A city opens a new tram line, and a year later house prices along it have risen. Did the tram do it? Maybe. Or maybe the city built the line where it already expected the area to boom. Or a new university opened nearby the same year. Every headline that says "X is linked to Y" hides this question, and most of the time the headline cannot answer it.
An economist sees the same pattern in a minimum-wage law, a school-class size, a tax change, a drug trial. The data show a link. The job is to say what the link would have been if nothing else had been going on. That is econometrics: statistics built for questions where you usually cannot run the experiment, and where the people in your data chose their own circumstances.
Take twelve students: hours studied, and the score each got. Plot them and the dots drift upward. A regression draws the one straight line that sits closest to all the dots. "Closest" has a precise meaning: for each dot, measure the vertical gap to the line (the residual), square it, and add up the squares. The line with the smallest total is the ordinary least squares (OLS) line. Squaring means one big miss counts far more than several small ones; the picture shows each squared miss as a literal square.
Twelve invented students. The best line is found by solving the least-squares equations; no other line has a smaller sum of squared misses.
The best line here is about 47 + 3.5 × hours (the readout gives it exactly). The coefficient b says: students who studied an hour more scored about 3.5 points higher. Notice what that sentence does not say. It compares different students. It does not say that making one student study an extra hour would raise her score by 3.5. That gap, between a comparison and a cause, is the rest of the course.
Suppose you regress wages on years of schooling in a big survey and find that each extra year goes with higher pay. Is that what schooling does? People with more drive or talent may both stay in school longer and earn more anyway. If so, the schooling coefficient is also soaking up the talent effect. This is omitted-variable bias, and it has a tidy rule:
In the lab below, the true effect of one more year of schooling is exactly 3 (thousand dollars). Talent also raises pay, and a dial sets how strongly talent pushes people toward more school. Switch the model between ignoring talent and including it.
True schooling effect = 3.0; talent adds 8 per standard deviation. Wage = 10 + 3·school + 8·talent + noise. "Randomise" means a coin-flip-like process assigns schooling regardless of talent.
Without JavaScript: with talent pushing schooling at strength 1.2, ignoring talent gives a schooling coefficient near 4.7 instead of the true 3.0; controlling for talent recovers about 3.0; randomising schooling also recovers about 3.0.
Ignoring talent at strength 1.2 gives about 4.7, over half again too large. Control for talent and it falls to about 3. The cost of this fix: you must measure the confounder. Most real confounders (motivation, family connections, how well-run the firm is) sit in no dataset. That is why economists spent decades arguing about what regressions could prove.
Press the third button. When schooling is assigned by a process unrelated to talent, the link between them is broken, and even the naive regression that ignores talent lands on 3. A coin flip is independent of everything: motivation, family, luck, things nobody thought to measure. That is why a randomised experiment is the gold standard, and why 2019's Nobel prize in economics went to Abhijit Banerjee, Esther Duflo and Michael Kremer for experimental approaches to reducing poverty.
But economists rarely get to flip the coin: you cannot randomly assign years of schooling, a recession or a minimum wage. The next three ideas are ways to find a coin that nature or a rule has already flipped. Together they are called the credibility revolution, a phrase from Joshua Angrist and Jörn-Steffen Pischke's 2010 essay, answering Edward Leamer's 1983 complaint that econometrics needed to "take the con out" of itself.
In 1992 New Jersey raised its minimum wage and neighbouring Pennsylvania did not. David Card and Alan Krueger surveyed fast-food restaurants in both states before and after (American Economic Review, 1994). Looking only at New Jersey before and after would confuse the law with anything else changing that year. Instead they used Pennsylvania as the stand-in for "what would have happened anyway". Their finding, that employment did not fall in New Jersey, contradicted the textbook prediction at the time. It has been disputed ever since: Neumark and Wascher re-analysed it with payroll records and found job losses, and the field's overall reading is still argued over. The design is not in dispute. Card shared the 2021 Nobel prize in part for it.
Treated group starts at 20, comparison at 24 (invented units). Dashed line: where the treated group would have gone without the policy, if it followed the comparison's trend.
With a common trend of +1.5 and no true effect, the treated group still rises by 1.5, so a naive "before and after" would claim a policy success. The difference-in-differences correctly says zero. The design's weak point is the third slider: it assumes the two groups would have moved in parallel without the policy. If the treated group was already on a different path, the estimate is off by exactly that gap. Good papers show the two lines tracking each other for years before the policy.
An instrumental variable is something that nudges the thing you care about but has no other route to the outcome. Angrist (1990) studied whether military service in the Vietnam era changed later earnings. Men who chose to serve differ from those who did not, so a plain comparison is confounded. But the draft lottery assigned numbers by birthday at random, and a low number made service much more likely. The lottery number is a coin that pushes people into uniform. Its effect on earnings, divided by its effect on service, isolates the effect of service for the people who served because of the lottery. His study found that service lowered earnings for the veterans concerned.
A worked number, invented to show the arithmetic: suppose 35% of low-number men served and 20% of high-number men, a first stage of 0.15; and low-number men earned $300 less a year on average. Then the estimated effect of serving is −300 ÷ 0.15 = −$2,000 a year. Dividing by a small first stage blows up noise, which is why weak instruments are dangerous.
A regression discontinuity uses a cut-off. Israeli school classes are capped at 40 pupils under a rule attributed to the twelfth-century scholar Maimonides, so a school with 40 pupils has one class and a school with 41 splits into two small classes of about 20. Angrist and Lavy (1999) compared schools just either side of those thresholds, where enrolment is close to a coin flip, to estimate what class size does to test scores. Slide the window below: narrow windows are fairer but noisier.
400 invented students; those scoring 60 or more get a support programme. Outcome rises 0.3 per score point anyway, so just comparing everyone above with everyone below overstates the jump.
Every estimate comes from one sample, and a different sample would have given a different number. The standard error is the typical size of that wobble. For a regression slope it shrinks with the square root of the sample size, grows with the noise around the line, and shrinks when x varies a lot:
The rule of thumb: an estimate more than about two standard errors from zero is called statistically significant at the 5% level. Significant means "hard to produce by sampling wobble alone if the true effect were zero". It does not mean big or important, and it does not mean true. Run 300 studies of a real but modest effect below and see who gets stars.
At n = 50 and a true slope of 0.3, a little over half of the studies reach significance, and those average about 0.40, a third above the truth, because only the lucky overestimates clear the bar. Drop to n = 10 and a true slope of 0.05 and the few stars (about 8%) average roughly 0.7, fourteen times too big. Published results are mostly the significant ones, so they exaggerate. That is one reason newer replications so often find smaller effects.
A test at the 5% level is wrong by chance 1 time in 20 when nothing is going on. Test 20 unrelated outcomes and the chance that at least one glows is 1 − 0.9520 = 64%. A researcher who tries several outcomes, subgroups or control sets and reports the one that "worked" is p-hacking, whether on purpose or just by wandering a garden of forking paths. Simmons, Nelson and Simonsohn (2011) showed how easy it is: using flexible, ordinary-looking choices they "found" that listening to a song about being older made students younger by about a year and a half. Economists are not exempt: Brodeur and co-authors (2016) found an unnatural pile-up of results just below the 0.05 line in top economics journals.
Each outcome compares two groups of 50 drawn from the same distribution, so every "effect" is noise. A cell lights up when |t| passes the 5% cut-off (1.98).
Defences: decide the analysis before seeing the data (pre-registration), report every test you ran, adjust the threshold when you run many, and above all prefer designs, like the ones above, where the answer does not depend on which controls you picked.
A regression gives you a comparison, and a comparison is only a cause if nothing else differs between the groups. Econometrics is a toolkit for making that true or nearly true (randomise, difference out a trend, use a lottery, zoom in on a cut-off), plus the humility to report how much your number would wobble and how many ways you tried.
Try this: the next time a headline says "X linked to Y", name one confounder that could drive both, and ask what coin flip, border or cut-off could separate them.
Sources. Least squares, omitted-variable bias, standard errors and the Wald estimator are standard textbook material (Stock & Watson; Wooldridge; Angrist & Pischke). Card & Krueger, "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review 84(4), 1994; the Nobel Prize in Economic Sciences 2021 went to David Card (half) and Joshua Angrist and Guido Imbens (half, "for their methodological contributions to the analysis of causal relationships"). Angrist, "Lifetime Earnings and the Vietnam Era Draft Lottery," 1990; Angrist & Lavy, "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," 1999; regression discontinuity was introduced by Thistlethwaite & Campbell in 1960. Angrist & Pischke, "The Credibility Revolution in Empirical Economics," Journal of Economic Perspectives 2010; Leamer, "Let's Take the Con out of Econometrics," 1983. Simmons, Nelson & Simonsohn, "False-Positive Psychology," 2011; Brodeur, Lé, Sangnier & Zylberberg, "Star Wars: The Empirics Strike Back," 2016. The 2019 economics Nobel went to Banerjee, Duflo and Kremer. Every number an interactive shows was computed in node from the code on this page: the best line is 47.0 + 3.53 × hours; omitted-variable bias at strength 1.2 is 8 × 1.2 ÷ (1.2² + 4) = 1.76 in theory (1.67 in this sample); 1 − 0.9520 = 64.2%. All datasets are invented.